Image processing apparatus, control method for image processing apparatus, and storage medium

ABSTRACT

An image processing apparatus that, in a case where processing corresponding to a user operation is determined to be cancelled before cancellation of the processing corresponding to the user operation, the user operation is determined to correspond to a hold-down operation, and a context menu is displayed in a case where the user operation corresponds to the hold-down operation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. application Ser. No.16/503,310 filed Jul. 3, 2019, which claims the benefit of priority fromJapanese Patent Application No. 2018-132479, filed Jul. 12, 2018, eachof which is hereby incorporated by reference herein in its entirety.

BACKGROUND Field of the Disclosure

The present disclosure generally relates to image processing and, moreparticularly, to an image processing apparatus that detects a useroperation on a touch display, a control method for the image processingapparatus, and a storage medium.

Description of the Related Art

In recent years, image processing apparatuses such as a multi-functionperipheral (MFP) including a touch display have been generally used. Atouch display has a configuration in which a display for providingdisplay during user operation and a touch panel that detects a positionwhere the user operation is performed are integrally formed.

An image processing apparatus includes applications for executingvarious functions (copy, scan, print, facsimile (FAX), etc.), andsoftware keys respectively corresponding to the applications aredisplayed on a touch display. A user taps a software key, thereby makingit possible to activate the application corresponding to the softwarekey.

To prevent execution of erroneous processing due to an error operationon the touch display, a condition for executing the processing, as wellas a condition for cancelling the execution of the processing can beset. Japanese Patent Application No. 2014-142729 discusses an imageprocessing apparatus having a configuration in which a predeterminedthreshold is set for a distance from a touch-down position on a buttonarea, and when the threshold is exceeded, processing corresponding to atouched-down button is not executed.

Not only a configuration in which processing corresponding to a softwarekey is executed when a user selects the software key, but also aconfiguration in which a plurality of processings corresponding to asoftware key is displayed as a context menu to be selected is known.Japanese Patent No. 6225959 discusses an information processingapparatus that displays a context menu in a range selected by a mouse.

SUMMARY

According to one or more aspects of the present disclosure, an imageprocessing apparatus includes a plurality of applications for usingfunctions of the image processing apparatus, a software key foractivating one of the plurality of applications being arranged on atouch display having a configuration in which a display and a touchpanel provided in the image processing apparatus are integrally formed,the image processing apparatus including a detection unit configured todetect a user operation based on a pressure on the touch display, and adisplay unit configured to display a context menu for displaying afunction corresponding to the software key on the touch display, whereinin a case where processing corresponding to the user operation isdetermined to be cancelled before cancellation of the processingcorresponding to the user operation, the user operation is determined tocorrespond to a hold-down operation, and the context menu is displayedin a case where the user operation corresponds to the hold-downoperation.

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a system configuration according to an exemplaryembodiment of the present disclosure.

FIG. 2 is a block diagram illustrating a hardware configuration of animage processing apparatus according to the exemplary embodiment of thepresent disclosure.

FIG. 3 is a block diagram illustrating a software configuration of theimage processing apparatus according to the exemplary embodiment of thepresent disclosure.

FIG. 4 illustrates an example of a home screen according to theexemplary embodiment of the present disclosure.

FIG. 5 illustrates examples of a context menu according to the exemplaryembodiment of the present disclosure.

FIGS. 6A and 6B each illustrates an example of an operation state on adisplay of the image processing apparatus according to the exemplaryembodiment of the present disclosure.

FIGS. 7A and 7B each illustrates an example of an operation state on thedisplay of the image processing apparatus according to the exemplaryembodiment of the present disclosure.

FIG. 8A illustrates an example of a variation at a touch position on anelectrostatic touch display, and FIG. 8B illustrates an example of avariation at a touch position on a pressure sensitive touch display.

FIGS. 9A, 9B, and 9C each illustrates an example of an operation stateon the display of the image processing apparatus according to theexemplary embodiment of the present disclosure.

FIG. 10 is a flowchart illustrating processing for displaying the homescreen according to the exemplary embodiment of the present disclosure.

FIG. 11 is a flowchart illustrating processing for determining a useroperation according to the exemplary embodiment of the presentdisclosure.

FIG. 12 is a flowchart illustrating processing for displaying the homescreen according to the exemplary embodiment of the present disclosure.

DESCRIPTION OF THE EMBODIMENTS

As discussed in Japanese Patent No. 6225959, it is possible to employ aconfiguration in which processing corresponding to a software key isexecuted by a tap operation on the software key and in which a contextmenu is displayed when the software key is selected by holding down thesoftware key. In addition, if Japanese Patent Application No.2014-142729 is taken into consideration in this configuration, it isalso possible to employ a configuration in which a context menu isdisplayed if a movement distance of a touch position touched by a userfalls within a certain distance, and the context menu is not displayedwhen the movement distance is greater than or equal to the certaindistance.

However, in a case where a pressure sensitive touch display is used, thecontext menu cannot be displayed depending on a user operation method,even when the user wishes to display the context menu. In the case ofusing the pressure sensitive touch display, the entire area with which auser's finger pad is in contact is recognized as a touch position by theuser. As a result, the width of the finger pad is regarded as themovement distance of the touch position. If the movement distancesatisfies the condition in which the context menu is not displayed, thecontext menu is not displayed even when the user holds down the softwarekey.

The present disclosure is directed to re-executing determination as towhether a user operation satisfies a condition for displaying a contextmenu, without cancelling processing of displaying the context menu, evenwhen the user operation on a software key displayed on a pressuresensitive touch display satisfies a condition for executing cancellationof the processing.

According to an exemplary embodiment of the present disclosure, it ispossible to re-execute the determination as to whether the useroperation satisfies the condition for displaying the context menu,without cancelling processing of displaying the context menu, even whenthe user operation on the software key displayed on the pressuresensitive touch display satisfies the condition for executingcancellation of processing.

Various exemplary embodiments, features, and aspects of the presentdisclosure will be described below with reference to the accompanyingdrawings. The following exemplary embodiments are merely examples, andtherefore are not intended to limit the present disclosure.

Definition of User Operation

A first exemplary embodiment will be described below. A user operationaccording to the present exemplary embodiment will be described below.

Touch (or touch operation): A general touch operation to be performed bya finger on a touch display.Release (or release operation): An operation in which a user's finger ina contact state is separated from the touch display.Tap operation (or tap): An operation in which a user's finger is causedto contact the touch display and is then released from the touchdisplay.Hold-down operation (or hold-down): An operation in which the statewhere a user's finger is in contact with the touch display is maintainedfor a certain period of time. A period in which the user's finger is incontact with the touch display is longer than that in the tap operation.Flick operation (or flick): An operation in which a user's finger ismoved to flick in any direction on the touch display.Drag operation (or drag): An operation in which a user's finger iscaused to move from a state where a specific object is selected by ahold-down operation, without executing the release operation.

System Configuration

FIG. 1 illustrates a system configuration of an image processingapparatus 101 according to the present exemplary embodiment. Specificexamples of the image processing apparatus 101 include a multi-functionperipheral (MFP) having various functions such as copy, scan, print, andfacsimile (FAX) functions.

Hardware Configuration of Image Processing Apparatus

FIG. 2 is a block diagram illustrating a hardware configuration of theimage processing apparatus 101.

A central processing unit (CPU) 111, a random access memory (RAM) 112, aread-only memory (ROM) 113, a static RAM (SRAM) 114, an input controlunit 115, a display control unit 116, an external memory interface (I/F)117, and a communication I/F controller 118, which are included in theimage processing apparatus 101, are connected via a system bus 110. Atouch panel 119, a display 120, an external memory 121, a scanner 122,and a printer 123 are each connected to the system bus 110. Eachprocessing unit included in the image processing apparatus 101 isconfigured to exchange data via the system bus 110.

The ROM 113 is a nonvolatile memory. Image data, data other than imagedata, various programs based on which the CPU 111 operates, and the likeare respectively stored in predetermined areas of the ROM 113. The RAM112 is a volatile memory and is used as a temporary storage area such asa main memory or a work area for the CPU 111.

The CPU 111, which may include one or more processors, one or morememories, circuitry, or a combination thereof, may control each unit ofthe image processing apparatus 100 by using the RAM 112 as a work memorybased on, for example, programs stored in the ROM 113. The programsbased on which the CPU 111 operates are not necessarily stored in theROM 113, but instead may be preliminarily stored in the external memory121 (such as a hard disk). The SRAM 114 is a nonvolatile recordingmedium capable of high-speed operation.

The input control unit 115 receives a user operation, generates acontrol signal, and supplies the control signal to the CPU 111. Forexample, the input control unit 115 receives a user operation from akeyboard (not illustrated), a mouse (not illustrated), or the touchpanel 119, which function as an input device. The touch panel 119 is aninput device configured to output coordinate information correspondingto a contact position on, for example, an input control unit that istwo-dimensionally or three-dimensionally configured. The presentexemplary embodiment will be described focusing on a case where thetouch panel 119 and the display 120 are integrated to form a touchdisplay 124. Further, according to the present exemplary embodiment, thetouch display 124 is a pressure sensitive touch display that determinesa user operation based on a pressure of a user's finger on the touchdisplay 124.

Pressure Sensitive Touch Display

A method for detecting a user operation on the touch display 124 will bedescribed with reference to FIGS. 8A and 8B.

FIG. 8A illustrates a state where the hold-down operation is executed bya user's finger in a case where the touch display 124 is anelectrostatic touch display. When the user intends to execute thehold-down operation, the pad of a finger 501 is pressed on the touchdisplay 124 as illustrated in FIG. 8A. As a result, a variation of thefinger 501 is detected as a coordinate variation 502.

FIG. 8B illustrates a state where the hold-down operation is executed bya finger in a case where the touch display 124 is a pressure sensitivetouch display. When the user intends to execute the hold-down operationby pressing the pad of the finger 501 on the touch display 124 asillustrated in FIG. 8B, the area of the pad of the finger 501 isregarded as coordinates where the hold-down operation is executed, andis detected as a coordinate variation. As a result, the coordinatevariation 503 is greater than the coordinate variation 502. Thecoordinate variation in the pressure sensitive touch display isdifferent from the coordinate variation in the electrostatic touchdisplay. This is because in the former case, a user operation isdetected based on a pressure applied when the touch display 124 ispressed, whereas in the latter case, a user operation is detected basedon a current that flows through the surface of the touch display 124 andfluctuates when the touch display 124 is pressed.

Referring again to FIG. 2, the CPU 111 controls each processing unit ofthe image processing apparatus 101 in accordance with programs based onthe control signal that is generated and supplied by the input controlunit 115 in response to the user operation performed on the inputdevice. Accordingly, the image processing apparatus 101 can perform anoperation depending on the user operation.

The display control unit 116 outputs a display signal for causing thedisplay 120 to display an image. For example, the CPU 111 supplies thedisplay control unit 116 with a display control signal generated basedon a program. The display control unit 116 generates a display signalbased on the display control signal and outputs the generated displaysignal to the display 120. For example, the display control unit 116causes the display 120 to display a graphical user interface (GUI)screen, which constitutes a GUI, based on the display control signalgenerated by the CPU 111.

In the present exemplary embodiment, the touch panel 119 is integrallyformed with the display 120, and also functions as an operation unit.For example, a manufacturer produces the touch panel 119 in such amanner that the transmittance of light is set so as not to inhibit thedisplay on the display 120, and the touch panel 119 is attached to anupper layer of a display surface of the display 120. Then, themanufacturer associates input coordinates on the touch panel 119 withdisplay coordinates on the display 120. The GUI that enables the user todirectly operate a screen displayed on the display 120 is thusconfigured. The touch panel 119 and the display 120 being integrated isreferred to as the touch display 124.

The external memory 121, such as a hard disk, a floppy disk, a compactdisc (CD), a digital versatile disc (DVD), or a memory card, can bemounted on the external memory I/F 117. Under the control of the CPU111, the external memory I/F 117 reads out data from the mountedexternal memory 121, and writes data into the external memory 121. Underthe control of the CPU 111, the communication I/F controller 118communicates with various networks 102 such as a local area network(LAN), the Internet, and wired or wireless networks. Variousapparatuses, such as a personal computer (PC), an MFP, a printer, and aserver, are connected to the networks 102 so that the apparatuses cancommunicate with the image processing apparatus 101.

The scanner 122 scans a document and generates image data. The printer123 executes print processing based on a user instruction input throughthe input control unit 115, or a command input from an externalapparatus via the communication I/F controller 118. The CPU 111functions as a specifying unit, and can specify, for example, a gestureoperation and a state as described below.

As the touch panel 119, various types of touch panels may be used.Examples of various types of touch panels include a resistive touchpanel, a capacitive touch panel, a surface acoustic wave touch panel, aninfrared touch panel, an electromagnetic induction touch panel, an imagerecognition touch panel, and an optical sensor touch panel.

The image processing apparatus 101 can store image data in the RAM 112or the external memory 121 by at least one of the storage methods. Forexample, the image processing apparatus 101 stores image data generatedbased on the document scanned by the scanner 122. Alternatively, theimage processing apparatus 101 stores image data received from anexternal apparatus, such as a PC, which is connected to the networks 102via the communication I/F controller 118. Yet alternatively, the imageprocessing apparatus 101 stores image data received from a portablestorage medium (such as a universal serial bus (USB) memory or a memorycard) mounted on the external memory I/F 117. In addition, the imageprocessing apparatus 101 may store image data in the external memory 121by another storage method. The image data to be stored may be data inwhich various setting contents including a print setting content on thedocument scanned by the scanner 122 are reflected. The image data to bedisplayed on the display 120 may include character information or imageinformation such as a photograph and a graphic image, or may includeboth character information and image information, or other information.Further, the image data may be sample images preliminarily stored.

Software Configuration of Image Processing Apparatus

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

FIG. 3 is a block diagram illustrating a software configuration of theimage processing apparatus 101. According to the present exemplaryembodiment, unless otherwise noted, information is exchanged betweensoftware modules illustrated in FIG. 3 in such a manner that the CPU 111controls the RAM 112, the ROM 113, the input control unit 115, and thedisplay control unit 116 via the system bus 110. The image processingapparatus 101 includes functions of a screen display control unit 301, ascreen input control unit 302, and a GUI parts control unit 303.

The screen display control unit 301 mainly performs applicationmanagement processing, processing for transferring information about ascreen to the display 120, and processing for sending a notificationabout an event input from the touch panel 119 to an application.

The screen input control unit 302 receives and analyzes a signal that isconverted from a user operation on the touch panel 119 by the inputcontrol unit 115, and converts the signal into an event format that canbe transmitted to an application. Then, the screen input control unit302 instructs the GUI parts control unit 303 to notify the input signal.

The GUI parts control unit 303 is a function for analyzing an eventreceived from the screen input control unit 302. The CPU 111 determinesthe configuration of the GUI parts control unit 303 and sends a displayinstruction to the screen display control unit 301.

Display on Display 120 of Image Processing Apparatus

Further features of the present disclosure will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

A home screen 401 to be displayed on the display 120 will be describedwith reference to FIG. 4. Unless otherwise noted, the home screen 401according to the present exemplary embodiment is displayed by the screendisplay control unit 301.

The home screen 401 includes a main area 402 in which a plurality ofsoftware keys is displayed, and a timeline 404 on which a plurality ofhistory buttons is displayed. The main area 402 is used to implementswitching to another main area 402 by a flick operation in a lateraldirection, and the timeline 404 enables switching of a history buttonthat is not displayed in FIG. 4 into a display state by a flickoperation in a longitudinal direction. However, a screen switchingmethod in each area is not particularly limited. On the timeline 404,for example, a history button that is not displayed can be displayed byperforming a lateral flick operation, or a display screen can beswitched to another display screen by performing a lateral flickoperation.

The flip operation and the drag operation can be detected as the useroperation on each screen.

In the main area 402 on the home screen 401, a plurality of softwarekeys for executing various functions of the image processing apparatus101 is displayed.

A copy button 406, which is an example of software keys, is a softwarekey for invoking a copy application. When a touch operation by a useroperation is detected by the input control unit 115, the display controlunit 116 causes the home screen 401 to shift to a copy setting screen(not illustrated).

A copy shared button 408 and a My transmission button 409 are softwarekeys created by a user (e.g., a user 1) who has logged in to the imageprocessing apparatus 101, or by an administrator. A person who hascreated a customized button determines whether to cause the customizedbutton to be constantly displayed on the home screen 401. For example,the copy shared button 408 is a custom button created by theadministrator of the image processing apparatus 101 and is constantlydisplayed on the home screen 401. When the copy shared button 408 ispressed, a document can be copied by 2-in-1 and double-sided printing.The My transmission button 409 is a custom button that is created by theuser (user 1) which has logged in to the image processing apparatus 101.When the log-in user is switched to another user, the My transmissionbutton 409 is not displayed. When the My transmission button 409 ispressed, document data can be transmitted in a portable document format(PDF) and with a resolution of 300 dpi.

FIG. 4 illustrates that eight software keys are displayed. Software keysthat cannot be displayed on the main area 402 are displayed on anothermain area 402 (not illustrated). The other main area 402 is switched anddisplayed using a lateral flick operation on the main area 402, or usinga switching bar 421.

A setting menu 407 is a menu in which the background of the home screen401, the size of each software key, or the number of software keys thatcan be displayed can be changed.

A right portion of the home screen 401 includes an upper software keyarea 403, the timeline 404, and a lower software key area 405. The uppersoftware key area 403 and the lower software key area 405 are areas inwhich keys that can be constantly displayed and executed are arranged.In the upper software key area 403 according to the present exemplaryembodiment, a home button for returning to the home screen 401 and alog-out button used for the user who has logged in to the imageprocessing apparatus 101 to log out. In the lower software key area 405,a stop button for cancelling processing performed by an application anda counter indicating the number of sheets used for print processing aredisplayed.

The timeline 404 is an area for displaying a history button generatedwhen processing corresponding to a software key displayed on the mainarea 402 is executed. In the history button displayed on the timeline404, the same processing contents are not displayed in an overlappingmanner For example, history buttons indicating the same document dataand the same print setting are not displayed on the timeline 404 in anoverlapping manner Specifically, when processing that is not present onthe timeline 404 is executed, the history button corresponding to theprocessing is added to the timeline 404 as a list for the first time. Anorder in which the history button is added is not particularly limited,and may be determined depending on the date and time when processing isexecuted.

Pressing a history button enables re-execution of processing based onthe contents of the processing previously executed. In the presentexemplary embodiment, a “copy” button, a “scan and transmit” button, a“scan and save” button are displayed as the history buttonscorresponding to the processing previously executed. The number ofhistory buttons that can be displayed on the timeline 404 is notparticularly limited.

FIG. 6A illustrates a state where a hold-down operation 418 is executedon the software key “scan and transmit” displayed on the main area 402,and a context menu 413 is displayed. A positional relationship betweenthe software key pressed to execute the hold-down operation 418 and thecontext menu 413 displayed as a result of pressing the software key isnot limited to the configuration illustrated in FIG. 6A. For example,the context menu 413 may be displayed in a lower right direction or anupper direction of the software key.

Even when the user's finger used to execute the hold-down operation in astate where the context menu 413 is displayed is released, the displaystate of the context menu 413 is maintained. When a function on thecontext menu 413 that is being displayed is selected from the state, theprocessing corresponding to the function is executed. On the other hand,when the user's finger is released in the state where the context menu413 is displayed and an area other than the context menu 413 in the homescreen 401 is touched, the context menu 413 that is being displayeddisappears.

One of the conditions for determining that the user operationcorresponds to the hold-down operation 418 is a time for which thedisplay 120 is pressed by the user operation. In the present exemplaryembodiment, assume that the time is 500 msec. In a case where thedisplay 120 is continuously pressed for 500 msec or longer, it isdetermined that the user operation corresponds to the hold-downoperation.

FIG. 7A illustrates a state where a hold-down operation 420 is executedon the history button “copy” on the timeline 404 and a context menu 417is displayed. Also, in the history key, the position where the contextmenu 417 is displayed is not particularly limited, like in the case ofsoftware keys. Assume that the condition for the time based on which itis determined that the user operation corresponds to the hold-downoperation 420 is 500 msec, like in the case of software keys.

FIG. 6B illustrates a case where a lateral drag operation (lateral drag419) is executed on the software key “scan and transmit”. In thismanner, in a case where the operation position is moved before thecondition for determining that the user operation corresponds to thehold-down operation is satisfied, the context menu 413 is not displayeddue to button execution cancellation processing. The button executioncancellation processing is processing for cancelling the execution ofprocessing corresponding to a pressed software key when the movementdistance from a position where the software key is touched to a positionwhere the user's finger is released is greater than or equal to apredetermined number of pixels. The button execution cancellationprocessing may be used because this processing prevents execution of anerroneous operation on the software key due to an erroneous dragoperation on the software key when the user executes the drag operationon the display 120 to switch the display to another main area 402.

FIG. 7B illustrates a state where a drag operation (longitudinal drag422) in the longitudinal direction is executed on the history button andthe context menu 417 is not displayed due to list execution cancellationprocessing. The list execution cancellation processing is processing forcancelling the execution of processing corresponding to a pressedhistory button when the movement distance from a position where asoftware key is touched to a position where the user's finger isreleased is greater than or equal to a predetermined number of pixels.The list execution cancellation processing may be used because thisprocessing prevents execution of an erroneous operation on the historykey due to an erroneous drag operation on the history key when the userexecutes the drag operation on the display 120 to cause the display 120to display the history button that has not been displayed.

Assume that a threshold for cancelling software key button executionprocessing is set to be greater than a threshold for cancelling historybutton list execution processing. For example, 35 pixels are set as thethreshold for the former case, and 45 pixels are set as the thresholdfor the latter case. In the areas coexisting on the home screen 401,different thresholds are used depending on the operation direction ofthe user operation in each of the areas.

As illustrated in FIG. 8B, in a case where the touch display 124 is apressure sensitive touch display, a coordinate variation correspondingto the area of a finger pad is detected. That is, since the finger padis long in the longitudinal direction, a coordinate variation is morelikely to occur in a longitudinal user operation than in a lateral useroperation, and thus the threshold for the movement distance on thetimeline 404 that involves the longitudinal user operation is set to begreater than that in the lateral user operation.

Context Menu

In each context menu, functions corresponding to held-down software keysare displayed as items. An “open” function and a “move button” functionare displayed in the context menu 413. When the “open” function isselected, a screen (not illustrated) for executing the processingcorresponding to the held-down software key is displayed on the display120. That is, selecting the “open” function in the context menu 413 issynonymous to pressing the corresponding software key.

On the other hand, when the “move button” function in the context menu413 is selected, all software keys displayed on the main area 402 are ina state of being movable by a user operation, and can be arranged at anylocation.

Other context menus will be described with reference to FIG. 5. Thedescriptions of the functions described above are omitted.

A context menu 414 is a context menu to be displayed when a sharedbutton is held down. The shared button is a customized button created bythe administrator and corresponds to the copy shared button 408described above. The context menu 414 includes a “call” function forexecuting a software key, a “change name of shared button” function forchanging the name of the shared button, a “delete shared button”function for deleting the shared button, and a “move button” function.

A context menu 415 is a context menu to be displayed when a My button isheld down. The term “My button” refers to a customized button created bya log-in user. The My button corresponds to the My transmission button409 described above. The context menu 415 includes the “call” function,a “change name of My button” for changing the name of the My button, a“delete My button” function for deleting the My button, and the “movebutton” function.

A context menu 416 is a context menu to be displayed when the settingmenu 407 is pressed. The context menu 416 includes the “move button”function, an “edit button” function, a “setting of button size andnumber of buttons” function, a “setting of background” function, a“setting of home screen management” function, and a “setting of shortcutbuttons for functions” function.

When the “setting of button size and number of buttons” function isselected, the size and the number of software keys displayed on the mainarea 402 can be changed. Although the eight software keys are displayedon the main area 402, for example, only six software keys may bedisplayed on the main area 402 by increasing the size of each button. Inthis case, the software keys that are not displayed may be displayed onanother main area 402 that is displayed by switching the switching bar421.

When the “setting of background” function is selected, the background ofthe main area 402 can be changed or set. When the “setting of homescreen management” function is selected, various management settings onthe home screen 401 can be executed. For example, display or non-displayof the timeline 404 can be set. The “setting of shortcut buttons forfunctions” function enables creation of a software key for executingprocessing desired by the user as a shortcut button on the main area402.

The context menu 417 is a context menu to be displayed when the historybutton is held down. The context menu 417 includes the “call” function,a “register on home screen” function, and a “delete” function. When the“register on home screen” function is selected, a button correspondingto processing corresponding to the held-down history button is displayedon the main area 402. The My transmission button 409 is an example of asoftware key created when a history button is held down. The historybutton displayed on the timeline 404 is associated with the user who haslogged in to the image processing apparatus 101. Accordingly, thecustomized button created when the “edit button” function in the contextmenu is pressed is displayed only when the user who has created thecustomized button logs in to the image processing apparatus 101.

Processing for Displaying Home Screen

FIG. 10 is a flowchart illustrating processing for displaying the homescreen 401 in the image processing apparatus 101 according to the firstexemplary embodiment. Each step illustrated in FIG. 10 is processing tobe performed in such a manner that the CPU 111 executes a programpreliminarily stored in the ROM 113 or the external memory 121. Theprocessing illustrated in FIG. 10 is executed when the image processingapparatus 101 is activated.

In step S802, the CPU 111 generates GUI parts such as a software key ora history button.

In step S803, the CPU 111 sets a threshold for a distance based on whichit is determined the execution of GUI parts is cancelled. In the presentexemplary embodiment, 10 pixels are set as a threshold for determiningthat pressing of a software key is cancelled (button executioncancellation processing), and five pixels are set as a threshold fordetermining that pressing of a history button is cancelled (listexecution cancellation processing).

In step S804, the CPU 111 sets a threshold for a distance based on whichit is determined that the hold-down operation on various buttons isexecuted. In the present exemplary embodiment, 35 pixels are set as thethreshold for the software key distance, and 45 pixels are set as thethreshold for the history button distance.

The threshold set in step S803 is different from the threshold set instep S804. This is because the user operation expected in step S803 isdifferent from the user operation expected in step S804. A conceivableuser operation in step S803 is, for example, the tap operation.Specifically, this operation can be an operation for displaying asetting screen by tapping a software key or a history button andexecuting processing corresponding to the software key or the historybutton. On the other hand, a conceivable user operation in step S804 isthe hold-down operation.

Since a time for which a user's finger is in contact with the touchdisplay 124 in the hold-down operation is longer than that in the tapoperation, a variation at a touch position is more likely to occur inthe hold-down operation. Further, in the tap operation, the user tendsto perform the operation by causing a user's fingertip to softly touchthe touch display 124, and thus the contact area is small On the otherhand, in the hold-down operation, the user causes the entire finger padto contact the touch display 124, and thus the contact area is large.Accordingly, the threshold for cancellation set in step S803 is set tobe smaller than the threshold set in step S804.

In step S805, the CPU 111 sets a threshold for a time based on which itis determined that the hold-down operation is executed. In the presentexemplary embodiment, assume that the threshold for the time based onwhich it is determined that the hold-down operation is executed onvarious buttons is 500 msec. In step S806, the CPU 111 displays the homescreen.

The processing for displaying the home screen 401 is as described above.The order in which steps S803 to 805 are executed is not particularlylimited, as long as various thresholds are set by the CPU 111.

Detection of User Operation on Pressure Sensitive Touch Display

Processing for detecting a user operation on a pressure sensitive touchdisplay will be described with reference to FIG. 11.

In step S808, the CPU 111 detects a touch of a finger on a software key.In the present exemplary embodiment, a user operation for causing auser's finger pad to contact the pressure sensitive touch display isreferred to as a touch, and the software key “scan and transmit” istouched. FIG. 9A illustrates a state where a hold-down operation 701 isexecuted on the software key “scan and transmit” on the home screen 401.A software key on which the hold-down operation is executed ishighlighted by, for example, changing the color of the software key soas to distinguish the software key from the other software keys. Thehighlighted state of the software key illustrated in FIG. 9A ismaintained until it is determined that the hold-down operation detectedin step 5808 corresponds to a cancel operation.

In step S809, the CPU 111 sets a timer for notifying a hold-down event,and starts counting the time from the detection of the user operation onthe touch display 124.

In step S810, the CPU 111 determines whether a move operation isdetected. The move operation is detected based on whether a touchposition is shifted from an initial touch position by the useroperation. The determination made in step S810 corresponds to branchprocessing for determining whether the user operation corresponds to thehold-down operation or the tap operation. In step S810, if the moveoperation is detected (YES in step S810), the processing proceeds tostep S811. If the move operation is not detected (NO in step S810), theprocessing proceeds to step S819.

In step S811, the CPU 111 determines whether an execution cancelthreshold is exceeded. If it is determined that the execution cancelthreshold is exceeded (YES in step S811), the processing proceeds tostep S812. In this case, the software key highlighted in step S808 isnot highlighted any more. FIG. 9B illustrates a state where theexecution of the hold-down operation 701 on a software key is cancelleddue to the occurrence of a coordinate variation in the hold-downoperation on the touch display 124. Assuming that a threshold for adistance based on which it is determined that the execution cancellationprocessing is executed is, for example, 10 pixels, a coordinatevariation of 10 pixels or more has occurred against the user's intentionduring the execution of the hold-down operation (701 to 702).

In step S811, if it is determined that the execution cancel threshold isnot exceeded (NO in step S811), the processing proceeds to step S813.

In a case where the execution cancel threshold is exceeded, in stepS812, the CPU 111 sets an execution cancel flag. The execution cancelflag indicates an ON/OFF switch in a program. When the execution cancelflag is set to the ON switch (TRUE), the execution cancel flag indicatesthat the execution of the processing is cancelled. The execution cancelflag set before step S812 is executed indicates FALSE. However, in acase where it is determined that the user's hold-down operation exceedsthe execution cancel threshold in step S811, the execution cancel flagis switched to TRUE.

In step S813, the CPU 111 determines whether the hold-down event hasalready been notified in step S812. If it is determined that thehold-down event has already been notified (YES in step S813), theprocessing proceeds to step S819. If it is determined that the hold-downevent has not been notified (NO in step S813), the processing proceedsto step S814.

In step S813, when the hold-down event is notified once, the result ofthe second and subsequent determinations in step S813 indicates “YES”.Then, the processing shifts to step S819 without carrying out theprocessing of step S815, and the timer set in step S809 is notcancelled. In other words, the hold-down event is not notified again andthe context menu is not repeatedly displayed until the release operationis detected in step S819.

In step S814, the CPU 111 determines whether the threshold for thedistance based on which it is determined that the hold-down operation isexecuted is exceeded. If the threshold for the distance based on whichit is determined that the hold-down operation is executed is exceeded(YES in step S814), the processing proceeds to step S815. If thethreshold for the distance based on which it is determined that thehold-down operation is executed is not exceeded (NO in step S814), theprocessing proceeds to step S816.

If it is determined that the threshold for the distance based on whichit is determined that the hold-down operation is executed is executed instep S814, it is determined that the user operation does not correspondto the hold-down operation. Accordingly, the CPU 111 executes thecancellation of the timer for notifying the hold-down operation, and instep S815, the CPU 111 resets counting by the timer.

In step S814, if it is determined that the threshold for the distancebased on which it is determined that the hold-down operation is executedis not exceeded (NO in step S814), the processing proceeds to step S816.In step S816, the CPU 111 determines whether the threshold for the timebased on which it is determined that the hold-down operation is executedis exceeded. In this case, the timer set in step S809 is referenced. Ifit is determined that the threshold for the time is exceeded (YES instep S816), the processing proceeds to step S817. If it is determinedthat the threshold for the time is not exceeded (NO in step S816), theprocessing proceeds to step S819.

In step S816, if the threshold for the time based on which it isdetermined that the hold-down operation is executed is exceeded (YES instep S816), the processing proceeds to step S817. In step S817, the CPU111 notifies the hold-down event. In step S818, the CPU 111 displays acontext menu. FIG. 9C illustrates a state of the home screen 401 in thiscase. Simultaneously with the display of the context menu, the softwarekey that has not been highlighted is highlighted again.

In step S819, the CPU 111 determines whether a release operation isdetected. If the release operation is detected (YES in step S819), theprocessing proceeds to step S820. If the release operation is notdetected (NO in step S819), the processing returns to step S810.

In step S820, the CPU 111 determines whether the execution cancel flagindicates TRUE. If the execution cancel flag is set to TRUE by theprocessing in step S812 (YES in step S820), this processing isterminated without executing the processing corresponding to thesoftware key touched in step S808. If the execution cancel flag does notindicate TRUE (NO in step S820), the processing proceeds to step S821.

In step S821, the CPU 111 executes the processing corresponding to thesoftware key touched in step S808.

As a result of pressing the pressure sensitive touch display 124 by theprocessing described above, the determination as to whether the useroperation corresponds to the hold-down operation is executed even whenthe execution of the processing corresponding to the user operation iscancelled. Consequently, it is possible to display a context menu evenwhen a coordinate variation occurs.

The processing illustrated in FIG. 11 is executed also when the useroperation on a software key does not correspond to the hold-downoperation and corresponds to the tap operation. If the tap operation isnot detected as the move operation, the processing shifts from step S810to step S819. After the tap operation is completed and the releaseoperation is detected in step S819, the processing of steps S820 andS821 is executed and the processing corresponding to the software key onwhich the tap operation has been executed is executed.

A second exemplary embodiment will be described below. The firstexemplary embodiment illustrates an example in which the touch display124 is a pressure sensitive touch display, while the second exemplaryembodiment illustrates a method for setting a threshold for determiningthe hold-down operation in consideration of a case where the touchdisplay 124 is an electrostatic touch panel.

FIG. 12 is a flowchart to be executed by the image processing apparatus101 when a multi-touch operation is performed. Each step illustrated inFIG. 12 is processed in such a manner that the CPU 111 executes aprogram stored in the ROM 113 or the external memory 121. The partsdescribed above are denoted by the same reference numerals, anddescriptions thereof are omitted.

After the distance threshold for cancelling the execution of thesoftware key is set in step S803, the CPU 111 acquires information aboutan operation unit panel (step S901). Examples of the information aboutthe operation unit panel include the type of the touch display, such asa pressure sensitive touch display, an electrostatic touch display, or atouch display other than the pressure sensitive touch display and theelectrostatic touch display, the number of inches, and a resolution.

In step S902, the CPU 111 sets a threshold for a distance based on whichit is determined that the hold-down operation is executed by using theinformation about the operation unit panel. A coordinate variation thatoccurs in the electrostatic touch display 124 is smaller than that inthe pressure sensitive touch display 124. Accordingly, the threshold forthe electrostatic touch display 124 is set to be smaller than that forthe pressure sensitive touch display 124. Specifically, the thresholdfor the pressure sensitive touch display 124 (threshold for the distancebased on which it is determined that the hold-down operation is executedon the main area 402) can be set to 35 pixels, while the threshold forthe electrostatic touch display 124 can be set to, for example, “15pixels”, which is smaller than the threshold for the pressure sensitivetouch display 124.

The processing for setting different thresholds depending on the type ofthe touch display has been described above. The above-describedprocessing makes it possible to display a context menu, regardless of acoordinate variation, even when an electrostatic touch display or apressure sensitive touch display is used as the touch display 124.

Other Embodiments

FIG. 11 illustrates a configuration in which the timer for notifying thehold-down event is not cancelled until the release operation is detectedin step S819. However, the present disclosure is not limited to thisconfiguration. A configuration for executing cancellation of the timerafter a context menu is displayed without the need for detecting therelease operation can also be employed. In this configuration, forexample, when the drag operation is performed to move the touch positionto the software key “My transmission” from the state where a hold-downoperation 703 illustrated in FIG. 9C is executed, the context menu 413can be switched to the context menu 415 corresponding to the softwarekey “My transmission”.

The first exemplary embodiment illustrates an example of theconfiguration in which one location on the pressure sensitive touchdisplay 124 is touched. In contrast, in a case where a plurality oflocations on the touch display 124 is touched, the processing flowillustrated in FIG. 8 is executed on the touch operation that is firstdetected. In other words, even when a plurality of locations on thetouch display 124 is touched and an operation other than the touchoperation that is first detected is detected as the touch operation, allthe operations are ignored (not output as an event).

In the first exemplary embodiment, the determination using the timethreshold is executed after the determination using the distancethreshold is executed, and it is determined whether the user operationcorresponds to the hold-down operation. However, the order in which thedeterminations using the thresholds are executed is not particularlylimited. Specifically, a configuration in which the determination as towhether the user operation corresponds to the hold-down operation ismade based on the time in step S814 and then the determination as towhether the hold-down operation is executed is made based on themovement distance in step S816 may be employed.

In the first and second exemplary embodiments, the present disclosurehas been described in detail above based on the preferred exemplaryembodiments. However, the present disclosure is not limited to thespecific exemplary embodiments, and various configurations within thescope of the disclosure are also included in the present disclosure.Some of the exemplary embodiments described above may be combined asneeded.

Further, the image processing apparatus 101 described above includesvarious apparatuses. Examples of the image processing apparatus 101include not only a PC, a personal digital assistant (PDA), and a mobilephone unit, but also a printer, a scanner, a FAX, a copying machine, anMFP, a camera, a video camera, and other image viewers.

Embodiment(s) of the present disclosure can also be realized by acomputerized configuration(s) of a system or apparatus that reads outand executes computer executable instructions (e.g., one or moreprograms) recorded on a storage medium (which may also be referred tomore fully as a ‘non-transitory computer-readable storage medium’) toperform the functions of one or more of the above-describedembodiment(s) and/or that includes one or more circuits (e.g.,application specific integrated circuit (ASIC)) for performing thefunctions of one or more of the above-described embodiment(s), and by amethod performed by the computerized configuration(s) of the system orapparatus by, for example, reading out and executing the computerexecutable instructions from the storage medium to perform the functionsof one or more of the above-described embodiment(s) and/or controllingthe one or more circuits to perform the functions of one or more of theabove-described embodiment(s). The computerized configuration(s) maycomprise one or more processors, one or more memories, circuitry, or acombination thereof (e.g., central processing unit (CPU), microprocessing unit (MPU), or the like), and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computerized configuration(s), for example, froma network or the storage medium. The storage medium may include, forexample, one or more of a hard disk, a random-access memory (RAM), aread only memory (ROM), a storage of distributed computing systems, anoptical disk (such as a compact disc (CD), digital versatile disc (DVD),or Blu-ray Disc (BD)™), a flash memory device, a memory card, and thelike.

While the present disclosure has been described with reference toexemplary embodiments, it is to be understood that the disclosure is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

1. An image processing apparatus, comprising: a touch panel display; adetection unit configured to detect a user operation based on a pressureof an instruction part on the touch display; a first determination unitconfigured to, in a case where a movement distance of the instructionpart touching a software key displayed on the touch panel displayexceeds a first threshold, determine a corresponding user operation asan operation for canceling selection of the software key; and a seconddetermination unit configured to, even if the movement distance of theinstruction part touching the software key exceeds the first threshold,determine a corresponding user operation as a hold-down operation in acase where a predetermined condition is satisfied, wherein thepredetermined condition includes at least a condition that the movementdistance of the instruction part does not exceed a second threshold thatis greater than the first threshold.
 2. The image processing apparatusaccording to claim 1, wherein the predetermined condition includes acondition that the touch panel display is touched with the instructionpart for predetermined time or longer.
 3. The image processing apparatusaccording to claim 1, wherein determination performed by the seconddetermination unit is repeated until the instruction part touching thesoftware key is released from the touch panel display.
 4. The imageprocessing apparatus according to claim 1, wherein, in a case where themovement distance of the instruction part exceeds the second threshold,timer count of touching the touch panel display with the instructionpart is reset.
 5. The image processing apparatus according to claim 1,further comprising: a display control unit configured to display a menuthat includes a list of processing for the software key in a case wherethe second determination unit determines the corresponding useroperation as the hold-down operation.
 6. The image processing apparatusaccording to claim 5, wherein the menu displayed by the display controlunit is a context menu.
 7. The image processing apparatus according toclaim 1, wherein, in a case where the instruction part is released fromthe touch panel display in a state in which the corresponding useroperation is determined by the first determination unit as the operationfor canceling selection of the software key, processing corresponding tothe software key is not performed.
 8. The image processing apparatusaccording to claim 1, wherein, even if the movement distance of theinstruction part exceeds the first threshold, in a case where thepredetermined condition is satisfied, the corresponding user operationis determined as the operation for canceling selection of the softwarekey and the hold-down operation.
 9. The image processing apparatusaccording to claim 1, wherein the detection unit detects a useroperation based on a pressure of the instruction part on the touch paneldisplay to determine a contact area in which the instruction part is incontact with the touch panel display.
 10. The image processing apparatusaccording to claim 1, wherein the touch panel display is a pressuresensitive touch panel display.
 11. The image processing apparatusaccording to claim 1, wherein, in a case where the second determinationunit determines the corresponding user operation as the operation forcanceling selection of the software key, display of the software keychanges.
 12. The image processing apparatus according to claim 1,wherein, even in a case where the first determination unit determinesthe corresponding user operation as the operation for cancelingselection of the software key, the second determination unit determineswhether the corresponding user operation is the hold-down operation ornot in a case where the instruction part is not released from the touchpanel display.
 13. A control method of an image processing apparatuswith a touch panel display, the method comprising: detecting a useroperation based on a pressure of an instruction part on the touch paneldisplay; in a case where a movement distance of the instruction parttouching a software key displayed on the touch panel display exceeds afirst threshold, determining a corresponding user operation as anoperation for canceling selection of the software key; and even if themovement distance of the instruction part touching the software keyexceeds the first threshold, determining a corresponding user operationas a hold-down operation in a case where a predetermined condition issatisfied, wherein the predetermined condition includes at least acondition that the movement distance of the instruction part does notexceed a second threshold that is greater than the first threshold. 14.A non-transitory storage medium that stores a program that causes acomputer to execute a control method of an image processing apparatuswith a touch panel display, the method comprising: detecting a useroperation based on a pressure of an instruction part on the touch paneldisplay; in a case where a movement distance of the instruction parttouching a software key displayed on the touch panel display exceeds afirst threshold, determining a corresponding user operation as anoperation for canceling selection of the software key; and even if themovement distance of the instruction part touching the software keyexceeds the first threshold, determining a corresponding user operationas a hold-down operation in a case where a predetermined condition issatisfied, wherein the predetermined condition includes at least acondition that the movement distance of the instruction part does notexceed a second threshold that is greater than the first threshold.